The Web in Theoretical Linguistics Research: Two Case Studies Using the Linguist’s Search Engine

0. Introduction The whisper of “does that sound ok to you?” is a familiar sound to most linguists: we often hear it in the audience when a presenter supports a theoretical point using a judgment of grammaticality or ungrammaticality, and someone in the audience disagrees with the judgment. In recent years a growing number of researchers have investigated the use of introspective judgments underlying work in theoretical syntax and in linguistics more generally. As Bard, et al. (1996) put it, each linguistic judgment is a “small and imperfect experiment.” Schütze (1996) and Cowart (1997) provide detailed discussions of instability and unreliability in such informal methods, which can lead to biased or even misleading results. Alternatives to introspective judgments include psychological methods (Bard et al. 1996), quantitative modeling in order to approximate judgments (Lapata et al. 2001), and broader reexamination of the idealizations underlying linguistic theory (Abney 1996; Sorace and Keller 2005). Tools for searching naturally occuring text potentially provide an additional window on linguistic data (Christ 1994; Corley et al. 2001; Blaheta 2002; Kehoe and Renouf 2002; König and Lezius 2002; Fletcher 2002; Kilgarriff, Evans et al. 2003), but typically these tools involve computational sophistication, restriction to a particular corpus, or shallow word-level search, making them less attractive to “the Ordinary Working Linguist without considerable computer skills” (Manning 2002). We designed the Linguist’s Search Engine (LSE) to empower ordinary working linguists to search the Web for linguistic data (http://lse.umiacs.umd.edu; Resnik and Elkiss 2005). The LSE’s architecture permits efficient search using syntactic and lexical criteria, with special attention to the needs of linguists, and